Beyond the regret minimization barrier: an optimal algorithm for stochastic strongly-convex optimization
نویسندگان
چکیده
We give a novel algorithm for stochastic strongly-convex optimization in the gradient oracle model which returns an O( 1 T )-approximate solution after T gradient updates. This rate of convergence is optimal in the gradient oracle model. This improves upon the previously known best rate of O( log(T ) T ), which was obtained by applying an online strongly-convex optimization algorithm with regret O(log(T )) to the batch setting. We complement this result by proving that any algorithm has expected regret of Ω(log(T )) in the online stochastic strongly-convex optimization setting. This lower bound holds even in the full-information setting which reveals more information to the algorithm than just gradients. This shows that any online-to-batch conversion is inherently suboptimal for stochastic strongly-convex optimization. This is the first formal evidence that online convex optimization is strictly more difficult than batch stochastic convex optimization.
منابع مشابه
Beyond the regret minimization barrier: optimal algorithms for stochastic strongly-convex optimization
We give novel algorithms for stochastic strongly-convex optimization in the gradient oracle model which return a O( 1 T )-approximate solution after T iterations. The first algorithm is deterministic, and achieves this rate via gradient updates and historical averaging. The second algorithm is randomized, and is based on pure gradient steps with a random step size. This rate of convergence is o...
متن کاملRegret Analysis for Continuous Dueling Bandit
The dueling bandit is a learning framework wherein the feedback information in the learning process is restricted to a noisy comparison between a pair of actions. In this research, we address a dueling bandit problem based on a cost function over a continuous space. We propose a stochastic mirror descent algorithm and show that the algorithm achieves an O( √ T log T )-regret bound under strong ...
متن کاملA Stochastic View of Optimal Regret through Minimax Duality
We study the regret of optimal strategies for online convex optimization games. Using von Neumann’s minimax theorem, we show that the optimal regret in this adversarial setting is closely related to the behavior of the empirical minimization algorithm in a stochastic process setting: it is equal to the maximum, over joint distributions of the adversary’s action sequence, of the difference betwe...
متن کاملEfficient Regret Minimization in Non-Convex Games
We consider regret minimization in repeated games with non-convex loss functions. Minimizing the standard notion of regret is computationally intractable. Thus, we define a natural notion of regret which permits efficient optimization and generalizes offline guarantees for convergence to an approximate local optimum. We give gradient-based methods that achieve optimal regret, which in turn guar...
متن کاملStochastic convex optimization with bandit feedback
This paper addresses the problem of minimizing a convex, Lipschitz function f over a convex, compact set X under a stochastic bandit feedback model. In this model, the algorithm is allowed to observe noisy realizations of the function value f(x) at any query point x ∈ X . The quantity of interest is the regret of the algorithm, which is the sum of the function values at algorithm’s query points...
متن کامل